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ABSTRACT 

We address the problem of finding patterns from multi- 
neuronal spike trains that give us insights into the multi- 
neuronal codes used in the brain and help us design better 
brain computer interfaces. We focus on the synchronous fir- 
ings of groups of neurons as these have been shown to play 
a major role in coding and communication ([6]). With large 
electrode arrays, it is now possible to simultaneously record 
the spiking activity of hundreds of neurons over large periods 
of time. Recently, techniques have been developed to effi- 
ciently count the frequency of synchronous firing patterns. 
However, when the number of neurons being observed grows 
they suffer from the combinatorial explosion in the number 
of possible patterns and do not scale well. In this paper, 
we present a temporal data mining scheme that overcomes 
many of these problems. It generates a set of candidate 
patterns from frequent patterns of smaller size; all possible 
patterns are not counted. Also we count only a certain well 
defined subset of occurrences and this makes the process 
more efficient. We highlight the computational advantage 
that this approach offers over the existing methods through 
simulations. 

We also propose methods for assessing the statistical sig- 
nificance of the discovered patterns. We detect only those 
patterns that repeat often enough to be significant and thus 
be able to automatically fix the threshold for the data-mining 
application. Finally we discuss the usefulness of these meth- 
ods for brain computer interfaces ([4l llljl. 
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1. INTRODUCTION 

Neurons form the basic computing elements of brain and 
hence, gaining the understanding of the coordinated behav- 
ior of groups of neurons is essential for gaining a principled 
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understanding of the brain function. Thus, one of the im- 
portant problems in neuroscience is that of understanding 
the functioning of a neural tissue in terms of interactions 
among its neurons . 

The neurons communicate with one another by means of 
voltage fluctuations called action potential or spikes. We 
can study the activity of a specific neural tissue by gath- 
ering data in the form of sequences of action potentials or 
spikes generated by each of a group of potentially intercon- 
nected neurons. Recent techniques, like Micro Electrode 
Array (MEA), imaging of ionic concentrations etc., have en- 
abled us in recording the activity of hundreds of neurons si- 
multaneously. Such recorded data, known as multi-neuronal 
spike train data, is a mixture of the stochastic spiking of ac- 
tivities of individual neurons as well as correlated spiking 
activity due to interactions or connections among neurons. 

One way to find out the interactions among neurons is to 
find patterns from the spike train data(t2j). The patterns 
help in understanding the relation between the spiking times 
of neurons which in turn can throw light on the interaction 
among neurons performing a specific function. 

Various algorrithms have been developed to find interest- 
ing patterns in spike train data. All the algorithms essen- 
tially find frequent (or less-frequent) occurences of specific 
patterns and try to establish the significance of their oc- 
curence in the data, that is, statistically show that these 
patterns have not occurred by chance and have occurred 
because of interaction among the constituent neurons. 

Predominantly, two kinds of patterns have been explored, 
namely, f . Sequential firing patterns 2. Synchronous firing 
patterns. Sequential firing patterns are used to represent a 
chain of neurons firing one after the other after a certain 
amount of delay in time. Such patterns have been found in 
data recorded from the hippocampus circuit. 

Synchronous firing patterns, on the other hand, represent 
a group of neurons firing very close to each other in time. 
The difference in the their spiking times is in the order of 
milliseconds. Some algorithms ([5]) use time binning tech- 
nique to find such patterns. The spiking times of neurons 
are binned into time intervals equal to time span of interest. 
Then the occurence of every possible pattern is checked in 
each time bin. Such binning techniques affect the time res- 
olution of the spike times. Also some patterns that occur 
across time bins will be missed. Recently techniques [13] 
have been developed that avoid time-binning and count the 
frequency of patterns more efficiently. However, these algo- 
rithms are all essentially correlation based and also count 
the frequency of all possible patterns. When the number of 



neurons being observed grows they suffer from the combina- 
torial explosion in the number of possible patterns and do 
not scale well. 

In this paper, we view this problem of finding synchronous 
patterns from a temporal datamining perspective. The pat- 
terns are represented as parallel episodes (with expiry times) 
discussed in the frequent episode discovery framework [12| . 
We use the parallel mining algorithm to discover frequent 
patterns whose frequency in the data is above a user speci- 
fied threshold. The algorithm is apriori based. It generates 
a set of candidate patterns from frequent patterns of smaller 
size. All possible patterns are not counted. Also we count 
only a certain well defined subset of occurrences and this 
makes the process more efficient. We highlight the compu- 
tational advantage that this approach offers over the existing 
methods through simulations. 

We also develop statistical techniques to establish the sig- 
nificance of the discovered patterns. 

The rest of the paper is organised as follows. Section [5] 
describes the parallel episode mining algorithm that we use 
to discover synchronous patterns. Section |3] presents a sig- 
nificance test for the parallel episodes. In Section 3] we 
present simulation results that shows the effectiveness of our 
method. Concluding remarks are provided in Section [S] 

2. FREQUENT EPISODE FRAMEWORK FOR 
DISCOVERY OF SYNCHRONOUS PAT- 
TERNS 

Temporal datamining is concerned with analyzing sym- 
bolic time series data to discover 'interesting' patterns of 
temporal dependencies ([ZHQ]). Recently we have proposed 
that some datamining techniques, based on the so called 
frequent episodes framework, are well suited for analyzing 
multi- neuronal spike train data [121 [3l [M]. Patterns of in- 
terest in spike data such as synchronous firings by groups of 
neurons, the sequential patterns, and synfire chains which 
are a combination of synchrony and ordered firings, can be 
efficiently discovered from the data using these datamining 
techniques. In this section we first briefly outline the fre- 
quent episodes framework and then qualitatively describe 
this datamining technique for discovering frequently occur- 
ring synchronous patterns. 

In the frequent episodes framework of temporal datamin- 
ing. the data to be analyzed is a sequence of events denoted 
by {{Ei,ti), (£2,^2), . . .) where Ei represents an event type 
and ti the time of occurrence of the i*** event. Ei's are drawn 
from a flnite set of event types, C,. The sequence is ordered 
with respect to time of occurrences of the events so that, 
ti < ti+i, Vi. The following is an example event sequence 
containing 11 events with 5 event types. 

{{A, 1), (B,3), {D,5), {A,5), (C,6), {A, 10), 

{E, 15), (B, 15), {B, 17), (C, 18), {C, 19)) ^ ' 

A parallel episode is an ordered tuple of event types. For 
example, {A B C) is a 3-node parallel episode. Such an 
episode is said to occur in an event sequence if there are 
corresponding events in the data sequence. In sequence ((l|, 
the events {{A, 1), {B, 3), (C, 6)} and {(B, 3), (C, 6), [A, 10)} 
constitute an occurrence of the parallel episode {ABC). 
We note here that occurrence of an episode does not require 
the associated event types to occur consecutively; there can 



be other intervening events between them. 

The objective in frequent episode discovery is to detect 
all frequent episodes (of different lengths) from the data. 
A frequent episode is one whose frequency exceeds a (user 
specifled) frequency threshold. The frequency of an episode 
can be defined in many ways. It is intended to capture 
some measure of how often an episode occurs in an event 
sequence. One chooses a measure of frequency so that fre- 
quent episode discovery is computationally efficient and, at 
the same time, higher frequency would imply that an episode 
is occurring often. In our algorithm we use the maximum 
non- overlapped occurences as the frequency measure. This 
definition of frequency results in very efficient counting algo- 
rithms with some interesting theoretical properties ([3 (3). 

In analyzing neuronal spike data, it is useful to consider 
methods, where, while counting the frequency, we include 
only those occurrences which satisfy some additional tem- 
poral constraints. Here we are interested in what we call 
expiry time constraint which is specified by giving a time 
span r. The constraint requires that span of occurence of 
the parallel episode is less that r. For example in sequence 
(lU, with an expiry time r = 5, the occurrence of parallel 
episode {(^4, 1), (B, 3), (C, 6)} is valid where as the occur- 
rence {(B, 3), (C, 6), (A, 10)} is not. As is easy to see, a 
parallel episode with expiry time constraints corresponds to 
what we called a synchronous pattern in the previous sec- 
tion. These are the temporal patterns of interest in this 
paper. To represent a parallel episode [A B C) with expiry 
time r we use the notation [A B C)t. 

Efficient algorithm to count such episodes exist ([12]). 
The algorithm counts the non-overlapped occurrence of an 
episode (say, (^4 B C)t ) as follows. While going down the 
data stream it remembers the latest time of occurrence of 
all its constituent events. Once all the events are seen at 
least once, it checks if the span of the latest occurrences of 
all the events is less than r. If the expiry time constraint is 
satisfied, then frequency counter is incremented and all the 
events are marked as not seen. The algorithm then proceeds 
further to look for more occurrences. It is easy to see that 
such a method counts only non-overlapped occurrences of 
the parallel episodes. 

However, an efficient counting algorithms alone is not suf- 
ficient. This is because at higher levels the number of paral- 
lel episodes to be counted increases exponentially. The prob- 
lem of exploding number of candidates is tackled through 
the classic apriori method that is popular in datamining. 
At each level, the number of parallel episodes that have to 
counted is generated from the frequent candidates at lower 
levels. 

Based on this idea, we have the following structure for 
the algorithm. We first get frequent 1-node episodes which 
are then used to make candidate 2-node episodes. Then, by 
one more pass over data, we find frequent 2-node episodes 
which are then used to make candidate 3-node episodes and 
so on. Such a technique is quite effective in controlling com- 
binatorial explosion and the number of candidates comes 
down drastically as the size increases. This is because, as 
the size increases, many of the combinatorially possible par- 
allel episodes of that size would not be frequent. This allows 
the algorithm to find large size frequent episodes efficiently. 
At each stage of this process, we count frequencies of not 
one but a whole set of candidate episodes (of a given size) 
through one sequential pass over the data. We do not actu- 



ally traverse the time axis in time ticks once for each pattern 
whose occurrences we want to count. We traverse the time- 
ordered data stream. As we traverse the data we remember 
enough from the data stream to correctly take care of all the 
occurrence possibilities of all episodes in the candidate set 
and thus compute all the frequent episodes of a given size 
through one pass over the data. The complete details of the 
algorithm are available in f|12]'). 

3. SIGNIFICANCE OF DISCOVERED SYN- 
CHRONOUS FIRING PATTERNS 

In the previous section we discussed effective algorithms to 
discover synchronous patterns. Here, we present significance 
tests to show that the obtained patterns are significant and 
have not occurred by chance. 

There have been many approaches for assessing the sig- 
nificance of detected firing patterns ([UIIS]). In the current 
analytical approaches, one generally employs a Null hypoth- 
esis that the different spike trains are generated by indepen- 
dent processes. In many cases one also assumes (possibly 
inhomogeneous) Bernoulli or Poisson processes. Then one 
can calculate the probability of observing the given number 
of repetitions of the pattern (or of any other statistic derived 
from such counts) under the null hypothesis of independent 
processes and hence calculate a minimum number of repeti- 
tions needed to conclude that a pattern is significant in the 
sense of being able to reject the null hypothesis. There are 
also some empirical approaches, which may be called the jit- 
ter methods, suggested for assessing significance. Here one 
creates many surrogate data streams from the experimen- 
tally observed data by perturbing (or jittering) the individ- 
ual spikes while keeping certain statistics same. Then, by 
calculating the empirical distribution of pattern counts on 
the sample of surrogate data, one assesses the significance 
of the observed patterns. 

Recently, significance tests for sequential patterns had 
been developed [T3]. These tests involve estimating the ex- 
pected frequency of serial episodes under a given null hy- 
pothesis by modelling the counting process of the algorithm. 
We also take a similar approach and modify the method to 
suit synchronous patterns. The Null hypothesis we assume 
is that all the neurons fire independently of each other. 

3.1 Modelling the counting process 

Suppose, we are operating at a time resolution of AT. 
(That is, the times of events or spikes are recorded to a 
resolution of AT). Then we discretize the time axis into 
intervals of length AT. For a parallel episode with expiry 
times, the span of any occurrence should be less than the 
expiry time, T (in steps of AT). But initially let us assume 
that the span of each occurrence of a parallel episode is ex- 
actly T time steps. (We can modify the counting to skip T 
time units once an occurrence is found.) Now the counting 
process explained in the previous section can be viewed in 
the following way. For each episode (say {A B C)t) whose 
frequency we want to find, we do the following. We start 
at time instant 1. We check to see whether there is an oc- 
currence of the episode starting from the current instant. If 
we find an occurrence we increment the counter and move 
ahead by T steps and again start looking for another occur- 
rence. If we do not find an occurrence we move ahead one 
step. We do this since we reach the end of the data stream 



of length L. 

Let p be the probability that we find an occurence of an 
episode at any given time instant. Then from the above 
description of counting we can say that, from any given time 
instant we move ahead by T time units with a probability 
p and move ahead by one time unit with probability 1 — p. 
This leads to the recurrence relation, 

F[L,T,p) = {l^p)F{L-l,T,p)+p{l + F{L-T,L,p)) (2) 

where, F{L,T,p) is the expected frequency of a episode. 
The notation F{L, T,p) denotes that the mean is a function 
of L,T and p. 

The boundary conditions for this recurrence are: 

F{x,y,p) — Q, il X < y and Vp. (3) 

Similarly, the mean of the square of the frequency, G{L, T,p), 
of the episode can be obtained as 

G(L,r,p) = (i-p)G(L-i,r,p) + 
p(i + G(L-r,r,p) + 2F(L-r,r,p)) ^' 

Hence, the variance V{L,T,p) is, 

V{L,T,p)^G{L,T,p) - {F{L,T,p)f (5) 

Since the neurons fire independently of each other the 
probability of an occurence of an episode at any time in- 
stant is given by, 

•n-l 

P = p"(AT)"^(T-1)"-'-'t' (6) 

1=0 

where, p be the unconditional probability that a neuron 
fires at any given time instant. We obtain p by estimat- 
ing the average rate of firing for this neuron from the data 
(or we may know it from other prior knowledge). Using 
the value of p, we can calculate values of F{L,T,p) and 
V{L, T, p) from equations[21[3]and[Sl For a given type-I error 
e, using the Chebyshev inequality, the fr equency th reshold 
can then be obtained as F{L,T,p) + ky^V{L, T,p), where 
k is the smallest integer such that > ^- We use this 
frequency threshold for mining significant parallel episodes 
(synchronous firing patterns). 

By using this frquency threshold, we ensure that the chances 
of a random episode being reported as frequent is less than 
e. So for low values of e, we can confidentally say that the 
patterns reported as frequent are not random patterns. 

4. SIMULATION RESULTS 

In this section we compare the parallel episode mining 
algorithm with a popular existing tool, NeuroXidence, in 
terms of running times, scalability and false positive rates. 
NeuroXidence ([13]) is used to detect an excess or a lack of 
synchronous firing in spike train data. This is done by count- 
ing the number of occurrences of synchronous firing patterns 
satisfying a given expiry time. Unlike our non- overlapped 
counts, NeuroXidence counts all occurrences of a pattern. 
For example, for the pattern, {A B C)t any set of spikes 
of A,B and C that satisfy the time constraint is considered 
as an occurrence. NeuroXidence counts the frequency of all 
patterns that occur at least once in the data. The counting 
process is essentially a correlation based technique. 



L 


Avg. Run Time (s) 


F.P.R. 


PE 


NX 


PE 


NX 


50000 


0.2 


51 


15% 


31% 


100000 


0.375 


134 


21% 


47% 


200000 


0.8 


270 


48% 


79% 



Table 1: Comparison of NeuroXidence (NX) and 
Parallel Episode Mining Algorithm (PE) : Average 
running time (in seconds) and False Positive Rates 
(F.P.R. )comparison for varying data lengths (L) . 
(Parameters: p = 5 Hz, T = 5, Number of Neurons 
= 20.) 



NeuroXidence employs a non-parametric method to as- 
sess the significance of the observed counts. The Null hy- 
pothesis is that the patterns occur by chance. The estimate 
of the chance frequency under null hypothesis is obtained 
by generating surrogate data. Surrogate data is created by 
jittering the spikes of the neurons independently of one an- 
other. This way the temporal cross structure in the data 
is destroyed while retaining the auto-structure of the spike 
trains. For every trial of the data obtained, around 25 surro- 
gates are created. The patterns frequencies are found out in 
the surrogate data set. From the values so obtained, we get 
an empirical distribution of the chance frequencies. Using 
that the significance of the observed frequency counts are 
obtained. 

NeuroXidence is found to be very effective in finding syn- 
chronous firing patterns [13) . 

For the results provided in this section we use spike train 
data generated by using the Poisson simulator described in 
[14j . Each neuron is modelled as an inhomogenous poisson 
process. Strong interactions among neurons 

can be input to the simulator by means of conditional 
probabilities. For example, if we want the spiking of A at 
any time t to affect the spiking of B at time t + t, we rep- 
resent it by a conditional probality P{B /{A,t)) = p. Since 
our null hypothesis is of independence no strong connections 
are embedded into the simulator. The neurons fire indepen- 
dently of one another. We include correlated firing in the 
data by means of external stimulation, that is, we embed 
synchronous firing in the data generated by the simulator. 
Different sized patterns (upto 7 nodes) with various expiry 
times are embedded in the data. 

Effectiveness of both the methods are assessed with re- 
spect to the running times and false positive rates. The 
methods are tested for varying parameters like random fir- 
ing rate, different expiry times, different number of neurons. 
The running times and false positives rate reported for the 
parallel episode algorithm are average values obtained from 
100 realizations of the data. In case of NeuroXidence, the 
values are averaged over 20 iterations. The results are re- 
ported in Tables IH4I 

NeuroXidence requires input data from various trials. For 
our experiments we split a single long data into 20 portions 
and give it as an input. For better statistical analysis, the 
number of surrogates for determing the empirical probability 
distribution is set at 25. 

Both the methods were found to be very effective in min- 
ing the embedded patterns. All the patterns that are embed- 
ded in that data were discovered by the both the methods. 
However, the parallel episode mining algorithm has huge 
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Avg. Run Time (s) 


F.P.R. 


PE 


NX 


PE 


NX 


5 


0.38 


51 


22% 


31% 


10 


0.50 


309 


22% 


49% 



Table 2: Comparison of NeuroXidence (NX) and 
Parallel Episode Mining Algorithm (PE) : Average 
running time (in seconds) and False Positive Rates 
(F.P.R. )comparison for varying random firing fre- 
quency (p) . (Parameters: L = 50000, T = 5, Num- 
ber of Neurons = 20.) 

computational advantage over NeuroXidence (refer Table [T| 
Such difference in times are because the NeuroXidence cal- 
culates the frequencies of all possible patterns in the data. 
But the parallel episode mining algorithms uses an efficient 
level wise procedure to count candidates generated out of 
frequent sub-episodes. Also, the statistical test required for 
NeuroXidence requires it to find the frequencies of pattern 
in the surrogate data. If the number of surrogates is 25, then 
effectively NeuroXidence has to calculate the frequencies in 
data that is 25 times longer than the input data. This is 
the reason for the marked difference in running times of the 
algorithms. 

The change in expiry time of mining does not affect the 
running times of the episodes mining algorithms(see Ta- 
ble H]). 
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Avg. Run Time (s) 


F.P.R. 


PE 


NX 


PE 


NX 


20 


0.38 


51 


22% 


31% 


30 


0.44 


233 


27% 


49% 


40 


0.54 


1193 


40% 


59% 



Table 3: Comparison of NeuroXidence (NX) and 
Parallel Episode Mining Algorithm (PE) : Average 
running time (in seconds) and False Positive Rates 
(F.P.R. )comparison for varying number of partici- 
pation neurons (M). (Parameters: p = 5 Hz, T = 5, 
L = 50000) 



T 


Avg. Run Time (s) 


F.P.R. 


PE 


NX 


PE 


NX 


3 


0.38 


21 


29% 


23% 


5 


0.38 


51 


22% 


31% 


8 


0.37 


122 


15% 


51% 


10 


0.37 


189 


14% 


54% 



Table 4: Comparison of NeuroXidence (NX) and 
Parallel Episode Mining Algorithm (PE) : Average 
running time (in seconds) and False Positive Rates 
(F.P.R. )comparison for varying expiry times (T) . 
(Parameters: p = 5 Hz, L = 50000, Number of Neu- 
rons = 20.) 

The running time of NeuroXidence increases drastically 
with expiry times. Similar effects can also be seen because 
of increase in background firing rates(see Table The 
number of false positives increase because more and more 
patterns start occurring more than once. 

The increase in number of neurons has a huge effect on 



the running times of NeuroXidence(see Table (3]). Infact for 
a network with 40 neurons the running time is as high as 
20 minutes for only 50 sec data at 5 Hz. This is because 
of the exponential increase in the number of patterns to be 
counted. 

From Tables [2] and [4l it is clear that change in random 
firing frequency and expiry times does not affect the running 
times very much. The algorithm will be able to scale up for 
longer data with many interacting neurons. 

5. CONCLUSIONS 

Fast decoding of information-bearing patterns are critical 
to the success of brain-computer interfaces. Data mining 
approaches, combined with statistical significance tests that 
does not require a huge amount of surrogate data, may pro- 
vide some of the answers. In this paper we have presented 
an approach that significantly more efficient than existing 
methods and should lay the foundations for more efficient 
decoding of neural signals and hence achieve better brain- 
computer interfaces. 
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